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ABSTRACT 

The major goal of the reading program was to take 
children who were severely disabled readers and bring them to the 
third grade level over the course of eight weeks of intensive 
instruction. There were three basic components of the program: (1) 
Talking Typewriters (each child spent about twenty minutes each day 
on the Talking Typewriter) ; (2) tutoring by teacher aides; and (3) 
homework, which involved use of the Talking Page and the Voice 
Mirror. Seventeen fourth grade students, nine fifth grade students, 
and two sixth grade students were selected for the program. The 
students were pretested with the Stanford Achievement Test. Those who 
scored below 2.0 on the Stanford were recommended for informal 
testing. The same two tests were given as posttests either upon the 
completion of the program or at the end of the period of time when 
the children were no longer able to participate because of summer 
vacation. The statistical results indicated that the program did not 
bring each participating child up to the third grade level. 
Twenty-five children did not make substantial progress on either the 
formal or the informal tests. It was recommended that further tests 
of the reading program be conducted outside the Minneapolis public 
schoo Is . (Author/ WR) 
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Minneapolis Public Schools 

Evaluation of a Reading Program for 
Severely Retarded Readers 
Spring 1971 

Summary 



The Reading Program was designed to take children who 
were severely disabled readers or nonreaders and advance 
them to the third grade reading level after eight weeks of 
intensive instruction involving the use of Talking Type- 
writers, Talking Pages, Voice Mirrors, parent cooperation 
and tutoring by teacher aides. 



An evaluation of program results clearly indicated 
that the children involved in the program were those for 
whom it was designated. However, the program did not 
accomplish its major goal of bringing each child up to 
the third grade level. Recommendation was made that this 
approach to teaching reading should be tested under more 
controlled conditions, preferably in* another school system. 
Most students didn't really complete the entire program; 
this may not have been a fair test of the program. 



It should be noted that this program was not related 
to the Minneapolis Basic Skill Centers program which a 7 ' :> 
uses the Talking Typewriters. 
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Minneapolis Public Schools 

Evaluation of a Reading Program for 
Severely Retarded Readers 
Spring 1971 

On April ?, 1971 Minneapolis School officials were approached 
by Mr. Richard Kobler of the Responsive Environment Corporation (REC) 
with an idea for improving the teaching of reading. It was claimed 
that within a relatively short time children could reach the third 
grade level in reading, regardless of their starting point. 

Following brief but rather intensive discussions a decision 
was made to give the proposed program a tryout. Minneapolis per- 
sonnel had already had considerable experience with one basic com- 
ponent of the proposed program - The Talking Typewriter. Additionally, 
children to be involved in the tryout were so far behind their 
classmates that it was felt that only good could result from their 
participation. 

For these reasons, but mainly because there was hope that 
this new approach would reach children that other approaches had 
not reached, a go-ahead was given on April 5, 1971. 

Locally, the program became known as the Kobler Reading Program. 
(This program was not related to the Minneapolis Basic Skill Centers 
program, which also uses the REC Talking Typewriters. Positive 
reading gains for severely disabled readers have been demonstrated 
through the Basic Skill Centers 1 approach. 



Description of the Program 

The Reading Program was designed to take children who were severely 
disabled readers or nonreaders and advance them to the third grade 
reading level after eight weeks of intensive instruction. The program 
consisted of exposure to the Kobler Reading Program, along with assistance 
from teacher and aides. Children were given a Talking Page and a Voice 
Mirror to use in their homes. Parents were asked to see that the children 
spent from 30 to U5 minutes each night on homework and were given direc- 
tions of how to use the equipment. 

Basically there were three components to the program. First, was 
the Talking Typewriters. Each child spent about 20 minutes each day on 
the Talking Typewrite; The second component was tutoring by teacher 
aides. The third component was homework, which involved use of the 
Talking Page and the Voice Mirror . a 

Objective of the Program 
The major goal of the Reading Program was to take children who 
were severely disabled readers and bring them to the third grade level 
over the course of eight weeks of intensive instruction. Specifically, 
each child, upon completion of the program would read at the third grade 
level, assuming he were not brain damaged nor mentally retarded. (At 
times, initial discussions had referred to reaching the 3.5 grade equivalent 

a Talking Typewriter: A special typewriter, with audio components, exhibitors, 

and an automatic projection unit combined by a computer 
to present a programmed system which provides sight, 
touch and sound inputs. 

Talking Page: A series of booklets and records which present phonic 

principles of beginning reading. 

Voice Mirror: A cassette recorder with instant automatic playback. 



level, but this use was not consistent. It is definite, however, that 
a minimum of the third grade level was tho goal for each child in this 
program) . 

No specification was made as to which tests were to be u;;ed to define 
the third grade reading level. No specification of student selection pro- 
cedures for the project were made by program proponents other than that 
the children be disabled readers, but not brain damaged nor mentally 
retarded. It was implied that the reading program would be applicable 
to a wide variety of children with reading disabilities. Unfortunately, 
there was no clear understanding of what was meant by M the program." 
Initially, Minneapolis school personnel understood that the program would 
be completed when a child finished the forty-five lessons on the Talking 
Typewriter Program. Later, program proponents indicated that the program 
would not be completed until forty hours of instruction were finished. 
This lack of understanding is a crucial point since one of the claims 
made by Kobler Program representatives was that the program could be ex- 
pected to result in relatively slow initial learning with positively 
accelerated growth toward the end of the program. 

Selection of Students 
The number of students in the program was limited to 30 because 
of cost and other practical considerations. Two schools were selected 
by the Minneapolis School administration for involvement. These selec- 
tions were based on availability of space and the availability of the 
students in these schools for this kind of program. 



Initially grades U 9 5 and 6 were to be involved, btudents in Lheso 
t\\ree grades who were referred by their teachers were tested with the 
Stanford Reading Achievement Test form W, as a rough screen for selection 
purposes. All students who were at grade 2.0 or lower were given considera- 
tion for the program. 

Teachers at School A made referrals from grades k 9 5 and 6. On April 21, 
25 students were tested but none of them were found to meet the criteria 
for selection. All students topped the Brimary I test. A second group 
of students at this school was then tested with Primary IT and again none 
of the students were found to meet the selection criteria. It was then 
decided to test students at the third grade level. Paragraph Meaning and 
Word Reading sections of the Stanford were administered. Additional sections 
were not given since it was found that students were not making any correct 
responses or were getting very few items correct. In essence, these students 
were nonreaders. 

Similar procedures were followed at School B where testing began on 
April 22 in grades k 9 5, and 6. Only one or two Uth, 5th and 6th graders 
met the criteria. On April 26 third grade students referred by teachers 
were tested and selections were made on the basis of low scores on the 
Stanford Achievement Test. 

Following the selection of students on the formal reading tests, an 
informal reading test based on the American Book Company Reading series was 
administered by a nonschool consultant hired for this purpose. Students 
who scored in the second grade reader or above also were excluded from the 
program. One exclusion occurred at School A. 

The final sample involved in this program included 17 fourth year 
students, 9 fifth year students and two sixth year students. Information 
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was not available for two students. Most of the students wore placed at 
the third grade level (kindergarten plus three years) since somo fifth 
and sixth year students had repeated one or two grades. Exact «radc 
placements could not be made due to the nongraded program in School A. 
For all practical purposes, the students in this experimental program 
were third graders. All of these students had scored below 2.0 on 
paragraph meaning and all had scored below 2.3 on word reading on the 
Stanford Reading Achievement Test, None of the participating children 
had scored above level B on the informal reading tests and most of them 
were at the pre-prjmer level. 

No records were kept of sex, race or other background characteristics 
although it appears, from the names, that 21 boys and 9 girls were initially 
involved. 

Record file information indicated that 11 children were living in 
"normal" families and l6 were living in families broken by divorce, separation 
or death. No information was available on 3 children. 

Test Administration 
On the pretest, children were tested in groups of 5 on the Stanford 
Achievement Test. Information is lacking on testing procedures for students 
at School B. 

Children who scored 2.0 or below on the Stanford were recommended 
for informal testing. The Informal Reading Inventory published by the 
Minneapolis 1 Instructional Materials Center was used. This test used 
vocabulary from the American Book Company Reading series (ABC), the series 
recently adopted for use throughout the inner city schools of Minneapolis. 
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The same two tests were given as posttests. The posttests were 
given either upon completion of the experimental program (i.e. completion 
of the lessons) or at the end of the period of time when the children 
were no longer able to participate in the program because summer vacation 
had started. Some children were tested a "third time since they had been 
tested upon school closing because of the uncertainty of their continuance 
in the program. Subsequently it was found that some of these children 
could continue with the program and they were then tested when they completed 
their *+5th lesson or on July ?6th, whichever came first. 

Because of the limited exposure time, the program was extended to 
July 9th so that as many students as possible would have opportunity to 
complete the U5 lessons. 



The Program was designed to operate for 8 weeks. This design did 
not prove feasible due to the late date of agreement on conducting the experi- 
ment as well as to administrative and operational difficulties; in getting 
the program started. Equipment had to be ordered and machinery installed. 
These problems delayed the start of the program so that the first child 
was not actually involved until April 27 > 1971. Since school closed on 
June 12th, a complete eight week exposure would not have been possible 
withouc some revision in scheduling. An attempt was made to have children 
continue in the program even though school had closed and this attempt 
was successful in part. Nevertheless, the average number of days in 
attendance at the program was only 23. 

Perhaps of more importance, at least as far us program proponents 
are concerned, is the number of hours of exposure to the program. None 
of the children in the program were exposed to the full ^0 hours which 



Length of the Program 




program proponents had booed to see. Only two students kind m;>r«; th;»n 40 
hours exposure and two-thirds of the j?tudents had Less thun ha Lf the 
exposure time advocated by REC officials. Average (meun) exposure time 
was 17 hours and 58 minutes. 

Despite the limited exposure time, a number of students did finish 
the U5 lessons. Possibly these completions reflected the emphasis of 
Minneapolis project administrators on lesson completion as opposed to 
amount of exposure time; an emphasis based on lack of communication as 
to what constituted u "completed program." 

Eight of twenty-six students for whom this information wa« available 
completed the k 1 } lessons — or were working on the last lesson. Another k 
had completed at least ho hours. 

About 7 of 26 had completed less than l/2 the lessons (20 or under). 

Results 

Results clearly indicate that the children involved in the program 
were those for whom the program was designated, that is, they were severely 
disadvantaged readers since most of them were in third grade and were 
reading at a level about one and one-half grades below that level on the 
average. It is also obvious from Table 1, that the program did not 
accomplish its major goal of bringing each of these children up to the 
third grade level. 
Standardized Test Results 

The mean grade equivalent, based on raw scores, on the Stanford 
pretest was 1.6 in Word Reading for the children in the program. On the 
posttest, the mean grade equivalent was also 1.6. For Paragraph Meaning, 
the pretest grade equivalent was 1.5 and on the posttest the grade equivalent 
was 1.6. 
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Table I gives mean raw scores and estimated ^rade equivalents basud 
on mean raw scores for Word Reading and Paragrauh Meaning. In essence, 
the average child in this program was reading some place between the firs 
and second grade level on the Stanford Achievement measure at the beginni 
of the program and was at about the same level at the end of the program, 
although some gain v/as evidenced in the average raw score. 

Table 2 shows individual results for the two schools. School A 
students started at a lower level on both Paragraph Meaning and Word 
Reading. They made some gains in Paragraph Meaning but remaineu at 
approximately the same level on Word Reading. Students at School B lost 
ground on Paragraph Meaning knd remained at approximately the same grade 
equivalent level on Word Reading. Actual Ly, students at School B had a 
lower average raw score on the paragraph meaning and word reading post- 
test than. they had on the pretests. 

Table 3 shows the number of individual students who gained, lost, or 
stayed the same on the two sections of the Stanford Achievement Tests, 
Fifteen of the twenty-six students on. whom pre- and post test measures 
were available on Word Reading showed gains while ten lost ground and one 
student stayed at exactly the same level. On Paragraph Meaning twelve 
students gained, eleven lost and two stayed the same. In essence, we 
have roughly the same number of students losing ground as we have making 
gains. 

Four students were given a second posttest on Word Reading ar> they 
remained in the program longer than anticipated and thus had additional 
exposure* Six students, similarily, were given a second posttest on 
Paragraph Meaning following additional exposure to the program. Mean 
raw scores and grade equivalents did not change substantially when the 



Table L 



.Pre- and Posttest Mean Raw Scores and Grade Equivalents on the 

Stanford Achievement Test 
Form W, Primary 1 





Word Reading 
N=26 


Paragraph 


Meani ng 




Raw Score 


Grade 
Equivalent 


Raw Score 


Grade 
Equivalent 


Pretest 


.16.88 


1.6 


IO.96 


1.5 


First 
Posttest 


17.31 


1.6 


12.52 


1.6 


Second 0 
Posttest 


18. 


1.7 


12.88 


1.6 



a 

Four students were given a second posttest on Word Reading 
and six students were given a second posttest on Paragraph 
Meaning. At the time of the first posttest it was believed 
that these students would not continue with the program. 
Some practice effect may have occurred because of this addi- 
tional testing. 
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Table 2 



Pre- and Posttest Mean Raw Scores and Grade 
Equivalent (G.E.) on the Stanford 
Reading Achievement Test, Form W 
Primary I for Students in Two 
Project Schools 



WORD 




A 


OLI1UUJ. 










READING 


Mean 




Mean 






Raw Score 


G.E. 


Raw Score * 


G.E. 


rre ues u 








A. ( 


Posttest 1 


15.31 


1.5 


19.31 


1.7 


a 

Posttest 2 


16.15 


1.6 


20.92 


1.8 


PARAGRAPH 


School 


A 


School 


B 






MEANING 


Mean 




Mean 






Raw Score 


G.E. 


Raw Score 


G.E. 


Pretest 


5.5^ 


1.3 


16.83 


1.7 


Posttest 1 


9.62 


1.5 


15.67 


1.6 


a 

Posttest 2 


10. 5k 


1.5 


15.1*2 


1.6 



a 

Four students were given a second posttest on Word Reading 
and six students were given a second posttest on Paragraph 
Meaning. At the time of the first posttest it was believed 
that these students would not continue with the program. 
Some practice effect may have occurred because of this addi- 
tional testing. 



10 



Table 3 



Number of Students Gaining, Losing, or Receiving the 
Same Score from Pretest to Posttest on Word 
Reading and Paragraph Meaning of the Stanford 
Achievement Test, Form W, Primary 1 and the 
ABC Informal Reading Test 



Gained 



Lost 



Stayed 
the 
Same 



Total" 
Number of 
Students 



Word 
Reading 



15 



10 



26 



Paragraph 
Meaning 



12 



11 



25 



ABC 

Informal 



Ik 



15 



29 



Number varies because all students were not present for 
all testing sessions 

'Twelve students scored at the lowest possible level (0) 
on the pretest and posttest. No losses were possible 
for this group 



Table k 

Distribution of Pretest and Posttest of the ABC 
Informal Reading Test Placement for 
Students in an Experimental Reading 
Program - Summer 1971 



Informal 
Reading 

Test Level Pretest Posttest 

0 12 6 

A 56 

B 11 

C 3 •» 

D 6 7 

E 2 l < 

F 0 0 

G 0 1 

Number 29>p m 29 



second posttest was used as the measure of gain instead of the first test. 
Slight gains registered from first to second post tests may have been due 
to practice effects. 

Informal Reading Test Results 

Table 3 also shows gains and losses for students on the ABC informal 
reading test. Fourteen students gained, no students lost ground, and fifteen 
stayed at the same level as on their pretest. However, since twelve students 
scored at level 0 on the pretest, it was impossible for them to lose ground. 

Table h shows the distribution of students on the Informal Reading Test 
at the beginning and end of the program. 

Twenty-one children were at the pre-primer level (O-A-B-L) on the 
pretest. At the time of the posttest, seventeen children were still at 
the pre-primer level. 

Further analysis showed that 15 of the 29 children tested had not 
gained a single level. Seven students gained one level, four gained two 
levels, two gained three levels and one student gained four levels. 

One student in school B gained two or more levels while six students 
in school A gained two or more levels. 

In short, it appears that we have random gains and losses on both 
the formal and informal reading inventories. 




Conclusions 

The statistical results are quite conclusive. This project did not 
reach its goal of bringing each participating child up to the third grade 
level in reading. Expert opinion, based on analysis of the Informal Reading 
Tests suggest that four children made substantial reading progress. Twenty- 
five children did not make substantial progress on either the formal or 
informal tests, although the test administrators notes suggest that some 
unmeasured gains may have taken place (e. g. attitude improvement). 

While the results of the program do not appear subject to question, . 
there is a question of whether or not the program was given an adequate test 
of its efficacy. 

In some major respects the program operations did not fulfill the 
expectations of REC officials. Average student exposure time was less than 
half the time advocated. Fewer than half the students completed the k$ 
lessons in the program. 

It should be noted, however, that those students who completed the 1*5 
lessons made no greater gains than students who completed fewer lessons. 
Students who spent the most time in the program fared no better than students 
with less time in the program. None of the students reached the third grade 
criterion, regardless of the amount of exposure to the program. (Results 
for individual students are given in the Appendix). 

Deviations from REC plans were due, in part, to imperfect communications 
between REC and Minneapolis School officials. The communications problem, 
in turn, was probably due to the extremely short planning and installation 
time which were available for initiating the program. 

In summary, it appears that the Kobler Reading Program was not given 
an adequate test due to insufficient planning time. This problem should not 
imply criticism of program planners, however, since it appears that the choice 

13 I?. 



was to try the Program under imperfect conditions, or not to try it at all. 
The rationale appeared to be that if the program succeeded great gains 
would have been made; if it failed, little would have been lost. 

Considering the conditions under which the program operated it must 
be concluded that most participating children did not substantially improve 
their reading. In view of their previous deficit in reading it is a point 
of conjecture as to whether or not they would have made any greater progress 
if they had remained in their original program for this brief period. 

Recommendations 

1. Limit experimental studies if adequate planning time is not available . 

Although in this case there was little choice about involvement in 
the program, and little or no cost to the school district, the experience 
does provide a practical example of the need for adequate time to spell 
out detailed objectives and conditions under which the program will operate. 

2. Further tests of the Kobler Reading Program should be conducted outside 
the Minneapolis Public Schools . 

This recommendation is an opinion based on teachers 1 and administrators 1 
comments. The climate for further experimentation with this approach does 
not appear favorable. In part, this climate is a result of the communications 
problem. 
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